{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Overview of `eolearn.core`\n",
    "\n",
    "`eolearn.core` is the main subpackage which implements the basic building blocks:\n",
    "\n",
    "- `EOPatch`,\n",
    "- `EOTask`,\n",
    "- `EONode`,\n",
    "- `EOWorkflow`,\n",
    "- `EOExecutor`,\n",
    "\n",
    "and commonly used functionalities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EOPatch <a id=\"EOPatch\"></a>\n",
    "\n",
    "The first basic object in the package is a data container, called `EOPatch`. \n",
    "\n",
    "![eopatch](./images/eopatch.png)\n",
    "\n",
    "- It is designed to store all types of EO data for a single geographical location.\n",
    "- The `EOPatch` can contain data (of the same location) for multiple times. If the `EOPatch` contains multiple collections of temporal data, they must have the same temporal axis (the images must correspond to the same time-points).\n",
    "- There is no limit to how much data a single `EOPatch` can store, but typically it shouldn't be more than the size of your RAM.\n",
    "\n",
    "Each `EOPatch` has an attribute `bbox` of type `sentinelhub.BBox` to define its area. The attribute `timestamps` defines the temporal component of an `EOPatch`, which is either `None` (for patches without a temporal dimension) or  a list of `datetime.datetime` objects.\n",
    "\n",
    "EO data can be divided into categories, called \"feature types\" according to the following properties:\n",
    "\n",
    "| `FeatureType` | Type of data | Time component |  Spatial component | Type of values | Python object | Shape |\n",
    "| --- | --- | --- | --- | --- | --- | --- |\n",
    "| DATA | raster | <span style=\"color:green\">yes</span> | <span style=\"color:green\">yes</span> | float | `numpy.ndarray` | `t x n x m x d` |\n",
    "| MASK | raster | <span style=\"color:green\">yes</span> | <span style=\"color:green\">yes</span> | integer | `numpy.ndarray` | `t x n x m x d` |\n",
    "| SCALAR | raster | <span style=\"color:green\">yes</span> | <span style=\"color:red\">no</span> | float | `numpy.ndarray` | `t x d` |\n",
    "| LABEL | raster | <span style=\"color:green\">yes</span> | <span style=\"color:red\">no</span> | integer | `numpy.ndarray` | `t x d` |\n",
    "| DATA_TIMELESS | raster | <span style=\"color:red\">no</span> | <span style=\"color:green\">yes</span> | float | `numpy.ndarray` | `n x m x d` |\n",
    "| MASK_TIMELESS | raster | <span style=\"color:red\">no</span> | <span style=\"color:green\">yes</span> | integer | `numpy.ndarray` | `n x m x d` |\n",
    "| SCALAR_TIMELESS | raster | <span style=\"color:red\">no</span> | <span style=\"color:red\">no</span> | float | `numpy.ndarray` | `d` |\n",
    "| LABEL_TIMELESS | raster | <span style=\"color:red\">no</span> | <span style=\"color:red\">no</span> | integer | `numpy.ndarray` | `d` |\n",
    "| VECTOR | vector | <span style=\"color:green\">yes</span> | <span style=\"color:green\">yes</span> | / | `geopandas.GeoDataFrame` | Required columns `geometry` and `TIMESTAMP` |\n",
    "| VECTOR_TIMELESS | vector | <span style=\"color:red\">no</span> | <span style=\"color:green\">yes</span> | / | `geopandas.GeoDataFrame` | Required column `geometry` |\n",
    "| META_INFO | anything | <span style=\"color:red\">no</span> | <span style=\"color:red\">no</span> | anything | anything | anything |\n",
    "\n",
    "Note: `t` specifies time component, `n` and `m` are spatial components (height and width), and `d` is an additional component for data with multiple channels.\n",
    "\n",
    "Let's start by loading an existing `EOPatch` and displaying it's content (i.e. features):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EOPatch(\n",
       "  bbox=BBox(((465181.0522318204, 5079244.8912012065), (466180.53145382757, 5080254.63349641)), crs=CRS('32633'))\n",
       "  timestamps=[datetime.datetime(2015, 7, 11, 10, 0, 8), ..., datetime.datetime(2017, 12, 22, 10, 4, 15)], length=68\n",
       "  mask_timeless={\n",
       "    LULC: numpy.ndarray(shape=(101, 100, 1), dtype=uint16)\n",
       "    RANDOM_UINT8: numpy.ndarray(shape=(101, 100, 13), dtype=uint8)\n",
       "    VALID_COUNT: numpy.ndarray(shape=(101, 100, 1), dtype=int64)\n",
       "  }\n",
       "  vector={\n",
       "    CLM_VECTOR: geopandas.GeoDataFrame(columns=['TIMESTAMP', 'VALUE', 'geometry'], length=55, crs=EPSG:32633)\n",
       "  }\n",
       "  label={\n",
       "    IS_CLOUDLESS: numpy.ndarray(shape=(68, 1), dtype=bool)\n",
       "    RANDOM_DIGIT: numpy.ndarray(shape=(68, 2), dtype=int8)\n",
       "  }\n",
       "  meta_info={\n",
       "    maxcc: 0.8\n",
       "    service_type: 'wcs'\n",
       "    size_x: '10m'\n",
       "    size_y: '10m'\n",
       "  }\n",
       "  scalar_timeless={\n",
       "    LULC_PERCENTAGE: numpy.ndarray(shape=(6,), dtype=float64)\n",
       "  }\n",
       "  scalar={\n",
       "    CLOUD_COVERAGE: numpy.ndarray(shape=(68, 1), dtype=float16)\n",
       "  }\n",
       "  vector_timeless={\n",
       "    LULC: geopandas.GeoDataFrame(columns=['index', 'RABA_ID', 'AREA', 'DATE', 'LULC_ID', 'LULC_NAME', 'geometry'], length=88, crs=EPSG:32633)\n",
       "  }\n",
       "  mask={\n",
       "    CLM: numpy.ndarray(shape=(68, 101, 100, 1), dtype=uint8)\n",
       "    CLM_INTERSSIM: numpy.ndarray(shape=(68, 101, 100, 1), dtype=bool)\n",
       "    CLM_MULTI: numpy.ndarray(shape=(68, 101, 100, 1), dtype=bool)\n",
       "    CLM_S2C: numpy.ndarray(shape=(68, 101, 100, 1), dtype=bool)\n",
       "    IS_DATA: numpy.ndarray(shape=(68, 101, 100, 1), dtype=uint8)\n",
       "    IS_VALID: numpy.ndarray(shape=(68, 101, 100, 1), dtype=bool)\n",
       "  }\n",
       "  label_timeless={\n",
       "    LULC_COUNTS: numpy.ndarray(shape=(6,), dtype=int32)\n",
       "  }\n",
       "  data_timeless={\n",
       "    DEM: numpy.ndarray(shape=(101, 100, 1), dtype=float32)\n",
       "    MAX_NDVI: numpy.ndarray(shape=(101, 100, 1), dtype=float64)\n",
       "  }\n",
       "  data={\n",
       "    BANDS-S2-L1C: numpy.ndarray(shape=(68, 101, 100, 13), dtype=float32)\n",
       "    CLP: numpy.ndarray(shape=(68, 101, 100, 1), dtype=float32)\n",
       "    CLP_MULTI: numpy.ndarray(shape=(68, 101, 100, 1), dtype=float32)\n",
       "    CLP_S2C: numpy.ndarray(shape=(68, 101, 100, 1), dtype=float32)\n",
       "    NDVI: numpy.ndarray(shape=(68, 101, 100, 1), dtype=float32)\n",
       "  }\n",
       ")"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "from eolearn.core import EOPatch\n",
    "\n",
    "INPUT_FOLDER = os.path.join(\"..\", \"..\", \"example_data\")\n",
    "INPUT_EOPATCH = os.path.join(INPUT_FOLDER, \"TestEOPatch\")\n",
    "\n",
    "eopatch = EOPatch.load(\n",
    "    INPUT_EOPATCH, lazy_loading=False  # Set this parameter to True to load data in memory only when first needed\n",
    ")\n",
    "\n",
    "eopatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are multiple ways how to access a feature in the `EOPatch`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(numpy.ndarray, (68, 101, 100, 13))"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from eolearn.core import FeatureType\n",
    "\n",
    "# All of these access the same feature:\n",
    "bands = eopatch.data[\"BANDS-S2-L1C\"]\n",
    "# or\n",
    "bands = eopatch[FeatureType.DATA][\"BANDS-S2-L1C\"]\n",
    "# or\n",
    "bands = eopatch[(FeatureType.DATA, \"BANDS-S2-L1C\")]\n",
    "# or\n",
    "bands = eopatch[FeatureType.DATA, \"BANDS-S2-L1C\"]\n",
    "\n",
    "type(bands), bands.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vector features are handled by `geopandas`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TIMESTAMP</th>\n",
       "      <th>VALUE</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2015-07-31 10:00:09</td>\n",
       "      <td>1.0</td>\n",
       "      <td>POLYGON ((465181.052 5080254.633, 465181.052 5...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2015-08-20 10:07:28</td>\n",
       "      <td>1.0</td>\n",
       "      <td>POLYGON ((465181.052 5080254.633, 465181.052 5...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2015-09-19 10:05:43</td>\n",
       "      <td>1.0</td>\n",
       "      <td>POLYGON ((465181.052 5080254.633, 465181.052 5...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2015-09-29 10:06:33</td>\n",
       "      <td>1.0</td>\n",
       "      <td>POLYGON ((465181.052 5080254.633, 465181.052 5...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2015-12-08 10:04:09</td>\n",
       "      <td>1.0</td>\n",
       "      <td>POLYGON ((465181.052 5080254.633, 465181.052 5...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            TIMESTAMP  VALUE  \\\n",
       "0 2015-07-31 10:00:09    1.0   \n",
       "1 2015-08-20 10:07:28    1.0   \n",
       "2 2015-09-19 10:05:43    1.0   \n",
       "3 2015-09-29 10:06:33    1.0   \n",
       "4 2015-12-08 10:04:09    1.0   \n",
       "\n",
       "                                            geometry  \n",
       "0  POLYGON ((465181.052 5080254.633, 465181.052 5...  \n",
       "1  POLYGON ((465181.052 5080254.633, 465181.052 5...  \n",
       "2  POLYGON ((465181.052 5080254.633, 465181.052 5...  \n",
       "3  POLYGON ((465181.052 5080254.633, 465181.052 5...  \n",
       "4  POLYGON ((465181.052 5080254.633, 465181.052 5...  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eopatch[FeatureType.VECTOR, \"CLM_VECTOR\"].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Special features are bounding box and timestamps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[datetime.datetime(2015, 7, 11, 10, 0, 8), datetime.datetime(2015, 7, 31, 10, 0, 9), datetime.datetime(2015, 8, 20, 10, 7, 28), datetime.datetime(2015, 8, 30, 10, 5, 47), datetime.datetime(2015, 9, 9, 10, 0, 17)]\n",
      "BBox(((465181.0522318204, 5079244.8912012065), (466180.53145382757, 5080254.63349641)), crs=CRS('32633'))\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"300\" height=\"300\" viewBox=\"465140.6625400123 5079204.501509398 1080.2586056234431 1090.5216788202524\" preserveAspectRatio=\"xMinYMin meet\"><g transform=\"matrix(1,0,0,-1,0,10159499.524697617)\"><path fill-rule=\"evenodd\" fill=\"#66cc99\" stroke=\"#555555\" stroke-width=\"7.27014452546835\" opacity=\"0.6\" d=\"M 465181.0522318204,5079244.8912012065 L 465181.0522318204,5080254.63349641 L 466180.53145382757,5080254.63349641 L 466180.53145382757,5079244.8912012065 L 465181.0522318204,5079244.8912012065 z\" /></g></svg>"
      ],
      "text/plain": [
       "<POLYGON ((465181.052 5079244.891, 465181.052 5080254.633, 466180.531 508025...>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(eopatch.timestamps[:5])\n",
    "print(repr(eopatch.bbox))\n",
    "\n",
    "eopatch.bbox.geometry  # draws the shape of BBox"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A list of all features in an `EOPatch` can be obtained with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(<FeatureType.DATA: 'data'>, 'CLP_S2C'),\n",
       " (<FeatureType.DATA: 'data'>, 'CLP'),\n",
       " (<FeatureType.DATA: 'data'>, 'NDVI'),\n",
       " (<FeatureType.DATA: 'data'>, 'BANDS-S2-L1C'),\n",
       " (<FeatureType.DATA: 'data'>, 'CLP_MULTI'),\n",
       " (<FeatureType.MASK: 'mask'>, 'CLM'),\n",
       " (<FeatureType.MASK: 'mask'>, 'IS_DATA'),\n",
       " (<FeatureType.MASK: 'mask'>, 'CLM_MULTI'),\n",
       " (<FeatureType.MASK: 'mask'>, 'CLM_INTERSSIM'),\n",
       " (<FeatureType.MASK: 'mask'>, 'IS_VALID'),\n",
       " (<FeatureType.MASK: 'mask'>, 'CLM_S2C'),\n",
       " (<FeatureType.SCALAR: 'scalar'>, 'CLOUD_COVERAGE'),\n",
       " (<FeatureType.LABEL: 'label'>, 'IS_CLOUDLESS'),\n",
       " (<FeatureType.LABEL: 'label'>, 'RANDOM_DIGIT'),\n",
       " (<FeatureType.VECTOR: 'vector'>, 'CLM_VECTOR'),\n",
       " (<FeatureType.DATA_TIMELESS: 'data_timeless'>, 'DEM'),\n",
       " (<FeatureType.DATA_TIMELESS: 'data_timeless'>, 'MAX_NDVI'),\n",
       " (<FeatureType.MASK_TIMELESS: 'mask_timeless'>, 'RANDOM_UINT8'),\n",
       " (<FeatureType.MASK_TIMELESS: 'mask_timeless'>, 'LULC'),\n",
       " (<FeatureType.MASK_TIMELESS: 'mask_timeless'>, 'VALID_COUNT'),\n",
       " (<FeatureType.SCALAR_TIMELESS: 'scalar_timeless'>, 'LULC_PERCENTAGE'),\n",
       " (<FeatureType.LABEL_TIMELESS: 'label_timeless'>, 'LULC_COUNTS'),\n",
       " (<FeatureType.VECTOR_TIMELESS: 'vector_timeless'>, 'LULC'),\n",
       " (<FeatureType.META_INFO: 'meta_info'>, 'maxcc'),\n",
       " (<FeatureType.META_INFO: 'meta_info'>, 'size_x'),\n",
       " (<FeatureType.META_INFO: 'meta_info'>, 'size_y'),\n",
       " (<FeatureType.META_INFO: 'meta_info'>, 'service_type')]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eopatch.get_features()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a new `EOPatch` and store some features inside."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EOPatch(\n",
       "  bbox=BBox(((0.0, 0.0), (1.0, 1.0)), crs=CRS('4326'))\n",
       "  timestamps=[datetime.datetime(2015, 7, 11, 10, 0, 8), ..., datetime.datetime(2017, 12, 22, 10, 4, 15)], length=68\n",
       "  mask_timeless={\n",
       "    NEW_MASK: numpy.ndarray(shape=(68, 10, 13), dtype=uint8)\n",
       "  }\n",
       "  data={\n",
       "    BANDS: numpy.ndarray(shape=(68, 101, 100, 13), dtype=float32)\n",
       "  }\n",
       ")"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "from sentinelhub import CRS, BBox\n",
    "\n",
    "# Since EOPatch represents geolocated data, it should always have a bounding box\n",
    "new_eopatch = EOPatch(bbox=BBox((0, 0, 1, 1), CRS.WGS84))\n",
    "\n",
    "new_eopatch[FeatureType.MASK_TIMELESS, \"NEW_MASK\"] = np.zeros((68, 10, 13), dtype=np.uint8)\n",
    "\n",
    "# If temporal features are added to an EOPatch that does not have timestamps (or if the dimensions do not match),\n",
    "# the user is warned that the EOPatch is temporall ill-defined\n",
    "\n",
    "new_eopatch.timestamps = eopatch.timestamps\n",
    "new_eopatch[FeatureType.DATA, \"BANDS\"] = eopatch[FeatureType.DATA, \"BANDS-S2-L1C\"]\n",
    "\n",
    "# The following wouldn't work as there are restrictions to what kind of data can be stored in each feature type\n",
    "# new_eopatch[FeatureType.MASK, 'NEW_MASK'] = np.zeros((10, 10, 13), dtype=np.uint8)\n",
    "# new_eopatch[FeatureType.VECTOR, 'NEW_MASK'] = np.zeros((10, 10, 13), dtype=np.uint8)\n",
    "\n",
    "new_eopatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to delete a feature:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EOPatch(\n",
       "  bbox=BBox(((0.0, 0.0), (1.0, 1.0)), crs=CRS('4326'))\n",
       "  timestamps=[datetime.datetime(2015, 7, 11, 10, 0, 8), ..., datetime.datetime(2017, 12, 22, 10, 4, 15)], length=68\n",
       "  data={\n",
       "    BANDS: numpy.ndarray(shape=(68, 101, 100, 13), dtype=float32)\n",
       "  }\n",
       ")"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del new_eopatch[FeatureType.MASK_TIMELESS, \"NEW_MASK\"]\n",
    "\n",
    "new_eopatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can save `EOPatch` into a local folder. In case an `EOPatch` already exists in the specified location, we have to allow to overwrite its features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from eolearn.core import OverwritePermission\n",
    "\n",
    "OUTPUT_FOLDER = os.path.join(\".\", \"outputs\")\n",
    "os.makedirs(OUTPUT_FOLDER, exist_ok=True)\n",
    "\n",
    "NEW_EOPATCH_PATH = os.path.join(OUTPUT_FOLDER, \"NewEOPatch\")\n",
    "\n",
    "new_eopatch.save(NEW_EOPATCH_PATH, overwrite_permission=OverwritePermission.OVERWRITE_FEATURES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load the saved version and compare it with original:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_eopatch = EOPatch.load(NEW_EOPATCH_PATH)\n",
    "\n",
    "new_eopatch == loaded_eopatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each `EOPatch` can be shallow or deep copied:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EOPatch(\n",
       "  bbox=BBox(((0.0, 0.0), (1.0, 1.0)), crs=CRS('4326'))\n",
       "  timestamps=[datetime.datetime(2015, 7, 11, 10, 0, 8), ..., datetime.datetime(2017, 12, 22, 10, 4, 15)], length=68\n",
       "  data={\n",
       "    BANDS: numpy.ndarray(shape=(68, 101, 100, 13), dtype=float32)\n",
       "  }\n",
       ")"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_eopatch.copy()\n",
    "new_eopatch.copy(deep=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EOTask <a id=\"EOTask\"></a>\n",
    "\n",
    "The next core object is `EOTask`, which is a single well-defined operation on one or more `EOPatch` objects.\n",
    "\n",
    "We can create a new EOTask by creating a class that inherits from the abstract `EOTask` class:\n",
    "\n",
    "```Python\n",
    "class FooTask(EOTask):\n",
    "    \n",
    "    def __init__(self, foo_param):\n",
    "        \"\"\" Task-specific parameters\n",
    "        \"\"\"\n",
    "        self.foo_param = foo_param\n",
    "        \n",
    "    def execute(self, eopatch, *, patch_specific_param):\n",
    "        \n",
    "        # Do what foo does on EOPatch and return it\n",
    "    \n",
    "        return eopatch\n",
    "```\n",
    "\n",
    "- In the initialization method we define task-specific parameters.\n",
    "- Each task has to implement the `execute` method.\n",
    "- `execute` method has to be defined in a way that:\n",
    "    * positional arguments have to be instances of `EOPatch`,\n",
    "    * other types of arguments should be keyword arguments.\n",
    "- Otherwise the task itself can do anything.\n",
    "\n",
    "Example of a task that adds a new feature to existing `EOPatch`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Any, Tuple\n",
    "\n",
    "from eolearn.core import EOTask\n",
    "\n",
    "\n",
    "class AddFeatureTask(EOTask):\n",
    "    \"\"\"Adds a feature to the given EOPatch.\n",
    "\n",
    "    :param feature: Feature to be added\n",
    "    :type feature: (FeatureType, feature_name) or FeatureType\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, feature: Tuple[FeatureType, str]):\n",
    "        self.feature = feature\n",
    "\n",
    "    def execute(self, eopatch: EOPatch, *, data: Any) -> EOPatch:\n",
    "        \"\"\"Returns the EOPatch with added features.\n",
    "\n",
    "        :param eopatch: input EOPatch\n",
    "        :param data: data to be added to the feature\n",
    "        :return: input EOPatch with the specified feature\n",
    "        \"\"\"\n",
    "        eopatch[self.feature] = data\n",
    "\n",
    "        return eopatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how such a task could be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EOPatch(\n",
       "  bbox=BBox(((0.0, 0.0), (1.0, 1.0)), crs=CRS('4326'))\n",
       "  timestamps=[datetime.datetime(2017, 1, 1, 0, 0), ..., datetime.datetime(2017, 5, 1, 0, 0)], length=5\n",
       "  data={\n",
       "    NEW_BANDS: numpy.ndarray(shape=(5, 100, 100, 13), dtype=float64)\n",
       "  }\n",
       ")"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eopatch = EOPatch(bbox=BBox((0, 0, 1, 1), CRS.WGS84), timestamps=[f\"2017-0{i}-01\" for i in range(1, 6)])\n",
    "\n",
    "add_feature_task = AddFeatureTask((FeatureType.DATA, \"NEW_BANDS\"))\n",
    "\n",
    "data = np.zeros((5, 100, 100, 13))\n",
    "\n",
    "eopatch = add_feature_task.execute(eopatch, data=data)\n",
    "\n",
    "eopatch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The majority of `eo-learn` consists of different EOTasks implementing different operations on EO data.\n",
    "\n",
    "The list of all EOTasks is available in the [documentation](https://eo-learn.readthedocs.io/en/latest/eotasks.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EONode and EOWorkflow <a id=\"EONode-and-EOWorkflow\"></a>\n",
    "\n",
    "EOTasks can be joined together into an acyclic processing graph called `EOWorkflow`. Since `eo-learn` `1.0` these tasks first have to be wrapped into instances of `EONode` class.\n",
    "\n",
    "Here is a simple example of how an `EOWorkflow` can be created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from eolearn.core import EONode, EOWorkflow, LoadTask, SaveTask\n",
    "\n",
    "new_feature = FeatureType.LABEL, \"NEW_LABEL\"\n",
    "\n",
    "load_task = LoadTask(path=INPUT_FOLDER)\n",
    "add_feature_task = AddFeatureTask(new_feature)\n",
    "save_task = SaveTask(path=OUTPUT_FOLDER, overwrite_permission=OverwritePermission.OVERWRITE_FEATURES)\n",
    "\n",
    "# Each EONode object defines dependecies to other EONode objects:\n",
    "load_node = EONode(load_task, inputs=[], name=\"Load EOPatch\")\n",
    "add_feature_node = EONode(add_feature_task, inputs=[load_node], name=\"Add a new feature\")\n",
    "save_node = EONode(save_task, inputs=[add_feature_node], name=\"Save EOPatch\")\n",
    "\n",
    "workflow = EOWorkflow([load_node, add_feature_node, save_node])\n",
    "# or\n",
    "workflow = EOWorkflow.from_endnodes(save_node)\n",
    "\n",
    "# Alternatively, a linear workflow could also be built with a helper function:\n",
    "# from eolearn.core import linearly_connect_tasks\n",
    "# nodes = linearly_connect_tasks(load_task, add_feature_task, save_task)\n",
    "# workflow = EOWorkflow(nodes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's display the dependency graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.43.0 (0)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"480pt\" height=\"44pt\"\n",
       " viewBox=\"0.00 0.00 480.36 44.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 40)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-40 476.36,-40 476.36,4 -4,4\"/>\n",
       "<!-- Load EOPatch -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>Load EOPatch</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"62.39\" cy=\"-18\" rx=\"62.29\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"62.39\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">Load EOPatch</text>\n",
       "</g>\n",
       "<!-- Add a new feature -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>Add a new feature</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"236.83\" cy=\"-18\" rx=\"76.09\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"236.83\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">Add a new feature</text>\n",
       "</g>\n",
       "<!-- Load EOPatch&#45;&gt;Add a new feature -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>Load EOPatch&#45;&gt;Add a new feature</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M125.16,-18C133.43,-18 142.02,-18 150.6,-18\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"150.64,-21.5 160.64,-18 150.64,-14.5 150.64,-21.5\"/>\n",
       "</g>\n",
       "<!-- Save EOPatch -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>Save EOPatch</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"410.62\" cy=\"-18\" rx=\"61.99\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"410.62\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">Save EOPatch</text>\n",
       "</g>\n",
       "<!-- Add a new feature&#45;&gt;Save EOPatch -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>Add a new feature&#45;&gt;Save EOPatch</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M313.18,-18C321.64,-18 330.24,-18 338.64,-18\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"338.77,-21.5 348.77,-18 338.77,-14.5 338.77,-21.5\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x7f9bdc7afc10>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "workflow.dependency_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`EOWorkflow` is executed by specifying `EOPatch`-related parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WorkflowResults(outputs={}, start_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 733751), end_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 961589), stats={'LoadTask-939b27aa45a511eeb8db-91a8de8b81da': NodeStats(node_uid='LoadTask-939b27aa45a511eeb8db-91a8de8b81da', node_name='Load EOPatch', start_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 733806), end_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 822464), exception_info=None), 'AddFeatureTask-939b2a9b45a511eea69d-e2612971e907': NodeStats(node_uid='AddFeatureTask-939b2a9b45a511eea69d-e2612971e907', node_name='Add a new feature', start_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 825206), end_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 825267), exception_info=None), 'SaveTask-939b2cb545a511eea722-ed1665ca815d': NodeStats(node_uid='SaveTask-939b2cb545a511eea722-ed1665ca815d', node_name='Save EOPatch', start_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 827230), end_time=datetime.datetime(2023, 8, 28, 15, 19, 54, 960678), exception_info=None)}, error_node_uid=None)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = workflow.execute(\n",
    "    {\n",
    "        load_node: {\"eopatch_folder\": \"TestEOPatch\"},\n",
    "        add_feature_node: {\"data\": np.zeros((68, 3), dtype=np.uint8)},\n",
    "        save_node: {\"eopatch_folder\": \"WorkflowEOPatch\"},\n",
    "    }\n",
    ")\n",
    "\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A result of a workflow execution is a `WorkflowResults` object. It contains information about times of each node execution and information about potential errors.\n",
    "\n",
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:**\n",
    "    \n",
    "A difference between executing tasks directly and executing tasks in a workflow is that in a workflow each `EOPatch` input object will be first shallow-copied before being passed to any task.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EOExecutor <a id=\"EOExecutor\"></a>\n",
    "\n",
    "`EOExecutor` handles the execution and monitoring of EOWorkflows. It enables executing a workflow multiple times and in parallel. At the end, it generates a report containing the summary of the workflow's execution process.\n",
    "\n",
    "Execute previously defined workflow with different arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 5/5 [00:00<00:00, 510.50it/s]\n"
     ]
    }
   ],
   "source": [
    "from eolearn.core import EOExecutor\n",
    "\n",
    "execution_args = [  # EOWorkflow will be executed for each of these 5 dictionaries:\n",
    "    {\n",
    "        load_node: {\"eopatch_folder\": \"TutorialEOPatch\"},\n",
    "        add_feature_node: {\"data\": idx * np.ones((10, 3), dtype=np.uint8)},\n",
    "        save_node: {\"eopatch_folder\": f\"ResultEOPatch{idx}\"},\n",
    "    }\n",
    "    for idx in range(5)\n",
    "]\n",
    "\n",
    "executor = EOExecutor(workflow, execution_args, save_logs=True, logs_folder=OUTPUT_FOLDER)\n",
    "\n",
    "results = executor.run(workers=3)  # The execution will use at most 3 parallel processes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make the report:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Report was saved to location: /home/ubuntu/Sinergise/eo-learn/examples/core/outputs/eoexecution-report-2022_02_09-12_38_30/report.html\n"
     ]
    }
   ],
   "source": [
    "executor.make_report()\n",
    "\n",
    "print(f\"Report was saved to location: {executor.get_report_path()}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
